Sensor fusion for measurement of physiological parameters

ABSTRACT

There is provided a system for measuring a physiological parameter of a person indicative of physiological pathology, comprising: a plurality of remote non-contact sensors, each of a different type of sensing modality, at least one hardware processor executing a code for: simultaneously receiving over a time interval, from each of the plurality of remote non-contact sensors monitoring a person, a respective dataset, extracting, from each respective dataset, a respective sub-physiological parameter of a plurality of sub-physiological parameters, analyzing a combination of the plurality of sub-physiological parameters, and computing a physiological parameter indicative of physiological pathology according to the analysis, wherein an accuracy of the physiological parameter computed from the combination is higher than an accuracy of the physiological parameter independently computed using any one of the plurality of sub-physiological parameters.

RELATED APPLICATION(S)

This application is a Continuation-in-Part (CIP) of U.S. patentapplication Ser. No. 16/923,132 filed on Jul. 8, 2020, the contents ofwhich are incorporated by reference as if fully set forth herein intheir entirety.

FIELD AND BACKGROUND OF THE INVENTION

The present invention, in some embodiments thereof, relates tomeasurements of physiological parameters and, more specifically, but notexclusively, to systems and methods for computing a physiologicalparameter indicative of physiological pathology.

During viral outbreaks, such as COVID-19 and/or flu, people are screenedto determine likelihood of the person suffering from respiratorpathology attributed to the viral outbreak. Sensors are used to helpscreen people for detecting likelihood of respiratory pathology. Forexample, a thermometer placed underneath a tongue of a person is used totake the temperature of a person to detect if the person has hightemperature indicative fever likely linked to respiratory pathology, ornormal range temperature. In another example, a pulse oximeter placed ona finger of a person measures heart rate and oxygen saturation (SpO2).High heart rate and/or low oxygen saturation are linked to respiratorypathology.

SUMMARY OF THE INVENTION

According to a first aspect, a system for measuring a physiologicalparameter of a person indicative of physiological pathology, comprises:a plurality of remote non-contact sensors, each of a different type ofsensing modality, at least one hardware processor executing a code for:simultaneously receiving over a time interval, from each of theplurality of remote non-contact sensors monitoring a person, arespective dataset, extracting, from each respective dataset, arespective sub-physiological parameter of a plurality ofsub-physiological parameters, analyzing a combination of the pluralityof sub-physiological parameters, and computing a physiological parameterindicative of physiological pathology according to the analysis, whereinan accuracy of the physiological parameter computed from the combinationis higher than an accuracy of the physiological parameter independentlycomputed using any one of the plurality of sub-physiological parameters.

According to a second aspect, a method of measuring a physiologicalparameter of a person indicative of physiological pathology, comprises:simultaneously receiving over a time interval, a respective dataset fromeach of a plurality of remote non-contact sensors each of a differenttype of sensing modality that are monitoring a person, extracting, fromeach respective dataset, a respective sub-physiological parameter of aplurality of sub-physiological parameters, analyzing a combination ofthe plurality of sub-physiological parameters, and computing aphysiological parameter indicative of physiological pathology accordingto the analysis, wherein an accuracy of the physiological parametercomputed from the combination is higher than an accuracy of thephysiological parameter independently computed using any one of theplurality of sub-physiological parameters.

According to a third aspect, a computer program product for measuring aphysiological parameter of a person indicative of physiologicalpathology, comprises: a non-transitory memory having stored thereon acode for executing by at least one hardware processor, comprisinginstructions for: simultaneously receiving over a time interval, arespective dataset from each of a plurality of remote non-contactsensors each of a different type of sensing modality that are monitoringa person, extracting, from each respective dataset, a respectivesub-physiological parameter of a plurality of sub-physiologicalparameters, analyzing a combination of the plurality ofsub-physiological parameters, and computing a physiological parameterindicative of physiological pathology according to the analysis, whereinan accuracy of the physiological parameter computed from the combinationis higher than an accuracy of the physiological parameter independentlycomputed using any one of the plurality of sub-physiological parameters.

In a further implementation form of the first, second, and thirdaspects, further comprising code for: prior to the extracting: detectingerrors according to an analysis of a combination of the datasets whentime synchronized, correcting and/or removing the detected errors fromthe datasets, wherein the extracting is performed for each respectivedataset for which the detected errors are corrected and/or removed.

In a further implementation form of the first, second, and thirdaspects, further comprising code for: prior to the extracting: detectingerrors according to an analysis of each respective dataset, excluding acertain dataset with detected errors from extraction of at least onecertain sub-physiological parameter, wherein the extracting excludesextracting the at least one certain sub-physiological parameter from theexcluded certain dataset.

In a further implementation form of the first, second, and thirdaspects, each sub-physiological parameter denotes a measurement of adifferent physiological manifestation of the person, and thephysiological parameter indicates a diagnosis of likelihood ofphysiological pathology, wherein none of the sub-physiologicalparameters when used independently are indicative of the diagnosis oflikelihood of physiological pathology.

In a further implementation form of the first, second, and thirdaspects, the plurality of sub-physiological parameters and correspondingremote non-contract sensors are selected from the group consisting of:(i) temperature computed from an analysis of a thermal images capturedby a thermal sensor, (ii) respiratory rate and/or breathing patterncomputed from an analysis of data outputted by a sensor selected fromthe group consisting of: a thermal sensor, a radar sensor, a Dopplersensor, and a short wave infrared sensor (SWIR) sensor, (iii) heart rateand/or heartbeat pattern computed from an analysis of data outputted bya sensor selected from the group consisting of: a radar sensor, aDoppler sensor, a thermal sensor, and a visual light sensor, (iv) oxygensaturation (SpO2) computed from an analysis of data outputted by asensor selected from the group consisting of: a SWIR sensor, and avisual light sensor, (v) one or more of: nasal congestion, sore throat,hoarse voice, and cough, obtained from an analysis of an acousticdataset outputted by an acoustic sensor.

In a further implementation form of the first, second, and thirdaspects, the subject is in a vehicle, and the respective datasets depictthe subject in the vehicle captured with a window of the vehicle open.

In a further implementation form of the first, second, and thirdaspects, further comprising code for generating instructions foradmitting the vehicle to a parking area when a value of thephysiological pathology is below a threshold.

In a further implementation form of the first, second, and thirdaspects, further comprising code for analyzing at least one of therespective datasets obtained from at least one of the plurality ofremote non-contact sensors for validating that a set of rules is met,and in response to the set of rules being met, performing theextracting, the analyzing the combination, and the computing thephysiological parameter, wherein the set of rules includes at least onerule selected from a group consisting of: a subject is in a vehicle, awindow of the vehicle is open, a location of the subject and/or vehicleis according to a target location, an engine of the vehicle is turnedoff, and vibrations of the vehicle are below a threshold.

In a further implementation form of the first, second, and thirdaspects, further comprising at least one of: (i) automaticallygenerating instructions to a controller that controls an automatic doorto open the door for admission of the person when the diagnosisindicates unlikelihood of physiological pathology, and to close the doorand/or maintain the door in the closed state to prevent admission of theperson when the diagnosis indicates likelihood of physiologicalpathology, and (ii) administering an effective treatment for treatmentof the physiological pathology, the treatment selected from the groupconsisting of: supplemental oxygen, antibiotics, anti-viral, mechanicalventilation, bronchodilators, and corticosteroids.

In a further implementation form of the first, second, and thirdaspects, one of the plurality of sub-physiological parameters comprise abreathing pattern, and one of the remote non-contact sensors comprise athermal sensor capturing a sequence of thermal images depicting an openmouth of the person, and further comprising code for: computing thebreathing pattern by analyzing changes in pixel intensity values ofpixels corresponding to a mouth cavity of the person in the sequence ofthermal images.

In a further implementation form of the first, second, and thirdaspects, each sub-physiological parameter denotes a different respectivemeasurement originating from a same single physiological manifestationof the person, and the physiological parameter indicative ofphysiological pathology is a single measurement of the same singlephysiological manifestation.

In a further implementation form of the first, second, and thirdaspects, a time interval of the datasets used for computing thephysiological parameter is shorter than a time interval of eachrespective dataset required to compute each respective sub-parameterwith an accuracy similar to an accuracy of the physiological parameter.

In a further implementation form of the first, second, and thirdaspects, the same single physiological manifestation indicative ofphysiological pathology is selected from the group consisting of: (i)respiratory rate and/or breathing pattern, (ii) heart rate and/orheartbeat pattern, and (iii) blood oxygen saturation (SpO2), and whereinthe datasets and corresponding remote non-contract sensors include twoor more sensors selected from the group consisting of: (i) thermalimages acquired by a thermal sensor, (ii) near infrared (NIR) imagesacquired by a NIR sensor, (iii) visual light images acquired by a visuallight sensor, and a (iv) dataset indicative of chest motion captured bya Doppler sensor and/or radar sensor.

In a further implementation form of the first, second, and thirdaspects, one of the plurality of sub-physiological parameters comprise abreathing pattern, and one of the remote non-contact sensors comprise athermal sensor capturing a sequence of thermal images depicting a faceof the person, and further comprising code for: computing the breathingpattern by analyzing changes in pixel intensity values of pixelscorresponding to nostrils and/or face mask of the person in the sequenceof thermal images.

In a further implementation form of the first, second, and thirdaspects, further comprising code for segmenting the nostrils and/or maskof the person in the sequence of thermal images by identifying regionsof changes in pixel intensity values, the change in pixel intensityvalues varies by an amount corresponding to a temperature change range,and a rate of change of the pixel intensity values corresponding to acandidate breathing rate range.

In a further implementation form of the first, second, and thirdaspects, the breathing pattern is computed as a breathing rate based ona time interval from maximal to maximal pixel intensity values of pixelscorresponding to the nostril and/or face mask of the sequence of thermalimages.

In a further implementation form of the first, second, and thirdaspects, analyzing changes in pixel intensity values comprises analyzingan average intensity value of intensity values of pixels depicting theface mask and/or nostrils.

In a further implementation form of the first, second, and thirdaspects, the change in pixel intensity values of pixels corresponding tothe face mask excludes pixels of the face mask corresponding to a noseof the person.

In a further implementation form of the first, second, and thirdaspects, the breathing pattern is computed by analyzing changes in pixelintensity values of pixels corresponding to a region on a face of theperson under a nose of the person in the sequence of thermal images.

In a further implementation form of the first, second, and thirdaspects, further comprising code for analyzing the sequence of thermalimages to identify a plurality of facial features and/or mask featuresindicative of regions of each thermal image depicting parts of a face ofthe person, wherein the breathing pattern is computed by analyzingchanges in pixel intensity values of pixels corresponding to nostrilsand/or face mask of the person according to the identified plurality offacial features and/or mask features.

In a further implementation form of the first, second, and thirdaspects, a first of the plurality of remote non-contact sensorscomprises a thermal and/or visual sensor capturing a sequence of thermaland/or visual images depicting a chest and/or head of the person, asecond of the plurality of remote non-contact sensors comprises aDoppler and/or radar sensor, and further comprising code for: analyzingat least one thermal and/or visual image to identify a target locationof the chest and/or head of the person, and generating instructions foradjustment of a steering mechanism for adjusting an orientation of theDoppler and/or radar sensor for capturing the dataset from theidentified target location, wherein a first sub-physiological parameteris extracted from a first dataset acquired by the thermal and/or visualsensor, and a second sub-physiological parameter is extracted from asecond dataset acquired by the Doppler and/or radar sensor.

In a further implementation form of the first, second, and thirdaspects, further comprising code for: analyzing a first dataset acquiredby a first remote non-contact sensor to obtain tracked locations on aplurality of fixed points on a head of the person, receiving a seconddataset depicting the head of the person acquired by a second remotenon-contact sensor, and correcting the second dataset using the fixedpoints of the first dataset, for tracking the plurality of fixed pointson the head of the person depicted in the second dataset.

In a further implementation form of the first, second, and thirdaspects, the extracting and the analyzing and the computing thephysiological parameter comprises obtaining an outcome of a classifiertrained on a training dataset including, for each of a plurality ofsubjects, a respective dataset acquired by each of the plurality ofremote non-contact sensors, and an associated label of the physiologicalparameter indicative of physiological pathology.

In a further implementation form of the first, second, and thirdaspects, at least one of the plurality of sub-physiological parameterscomprises a demographic parameter, and wherein the analyzing thecombination of the plurality of sub-physiological is according to abaseline defining physiological pathology according to the demographicparameter.

Unless otherwise defined, all technical and/or scientific terms usedherein have the same meaning as commonly understood by one of ordinaryskill in the art to which the invention pertains. Although methods andmaterials similar or equivalent to those described herein can be used inthe practice or testing of embodiments of the invention, exemplarymethods and/or materials are described below. In case of conflict, thepatent specification, including definitions, will control. In addition,the materials, methods, and examples are illustrative only and are notintended to be necessarily limiting.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

Some embodiments of the invention are herein described, by way ofexample only, with reference to the accompanying drawings. With specificreference now to the drawings in detail, it is stressed that theparticulars shown are by way of example and for purposes of illustrativediscussion of embodiments of the invention. In this regard, thedescription taken with the drawings makes apparent to those skilled inthe art how embodiments of the invention may be practiced.

In the drawings:

FIG. 1 is a flowchart of a method for measuring a physiologicalparameter of a person indicative of physiological pathology, inaccordance with some embodiments of the present invention;

FIG. 2 is a block diagram of components of a system for measuring aphysiological parameter of a person indicative of physiologicalpathology, in accordance with some embodiments of the present invention;

FIG. 3 is a graph for computation of sub-physiological parameter(s)generated from an analysis of multiple thermal images of a person, inaccordance with some embodiments of the present invention;

FIG. 4 is a thermal image of a chest used for computingsub-physiological parameter(s), in accordance with some embodiments ofthe present invention;

FIG. 5 is a graph created based on an output of a non-contact radarsensor sensing a chest of a person correlated with another graph createdbased on output of a contact belt sensor measuring the chest of theperson, in accordance with some embodiments of the present invention;

FIG. 6 includes thermal images that are analyzed to measure atemperature of an identified mask and/or measure a temperature of anidentified forehead of the respective depicted person, in accordancewith some embodiments of the present invention; and

FIG. 7 includes thermal images depicting a person walking, for whichfacial features are identified, represented by stars, in accordance withsome embodiments of the present invention.

DESCRIPTION OF SPECIFIC EMBODIMENTS OF THE INVENTION

The present invention, in some embodiments thereof, relates tomeasurements of physiological parameters and, more specifically, but notexclusively, to systems and methods for computing a physiologicalparameter indicative of physiological pathology.

An aspect of some embodiments of the present invention relates to asystem, an apparatus, a method, and/or code instructions (i.e., storedon a memory and executable by one or more hardware processors) formeasuring a physiological parameter of a person, optionally thephysiological parameter being indicative of physiological pathology, forexample, respiratory, cardiovascular, skin pathology, mental statepathology (e.g., delirium, dementia, drug induced), and the like. Therespiratory pathology is optionally caused by a viral infection, forexample, COVID-19, flu, and the like. Multiple datasets aresimultaneously received from respective remote, non-contact sensors,each of a different type of sensor modality, for example, thermalimages, visible light images, ultrasound, laser speckle imaging, Lidar,and radar. A respective sub-physiological parameter may be extractedfrom each respective dataset. A combination of the sub-physiologicalparameters is analyzed. Alternatively or additionally, a combination ofthe datasets is analyzed, for example, a correlation between the twodatasets. The combination of datasets may be analyzed withoutnecessarily extracting the sub-physiological parameter from eachdataset. A physiological parameter indicative of physiological pathologyis computed according to the analysis. The accuracy of the physiologicalparameter computed from the combination of sub-physiological parametersand/or combination of datasets is higher than an accuracy thephysiological parameter computed independently for each one of thesub-physiological parameters, and/or computed for a subset ofsub-physiological parameters that is smaller than the full set ofsub-physiological parameters from all sensors. In response to thecomputed physiological parameter, instructions may be automaticallygenerated, for example, for execution by a controller for automaticallyopening a door to enable entry into an enclosure for people that areunlikely infected by a viral disease. It is also possible that detectionof potential pathology will lead to further screening by additionalmethods. For example, a person suspected of respiratory pathology may besent for x-ray imaging, and/or a person suspected of cardiovascularpathology may be sent for ECG. In another example, the subject may belocated within a vehicle, with the datasets of the subject capturedwhile the subject is inside the vehicle, optionally with the windowsdown. Entry of the vehicle into a parking area (e.g., parking lot,garage) and/or further driving (e.g., crossing a border, entering acity, entering a toll highway, crossing a bridge, boarding a ferry) maybe denied when the subject is determined to have the physiologicalpathology such as according to a set of rules such as the value of thephysiological parameter being above a threshold (e.g., by automaticallygenerating instructions for execution by a control to activate amechanism to close a gate and/or keep the gate closed). Entry of thevehicle may be permitted when the subject is determined not to havephysiological pathology such as according to the set of rules such asthe value of the physiological parameter being below the threshold(e.g., by automatically generating instructions for execution by acontrol to activate a mechanism to open a gate).

Optionally, in a first implementation, each respective sensor representsa different physiological manifestation of the person. Differentsub-physiological parameters are computed for each respective datasetacquired by each respective sensor, for example, temperature is computedfrom thermal images outputted by a thermal sensor, and heart rate iscomputed from a radar sensor. The physiological parameter indicative ofphysiological pathology, for example, likelihood of being infected withthe viral disease, is computed from the combination of differentsub-physiological parameters indicative of different respectivephysiological manifestations.

Alternatively, in a second implementation, the respective datasetacquired by each respective sensor represents a similar (e.g., same)single physiological manifestation of the person. The same singlesub-physiological parameter may be computed for each respective datasetacquired by each respective sensor, for example, respiratory rate iscomputed from a combination of thermal images captured by a thermalsensor, and chest motion captured by a radar sensor. The physiologicalparameter indicative of physiological pathology, for example, likelihoodof being infected with the viral disease, is computed from thecombination of similar/same sub-physiological parameters indicative ofsimilar/same respective physiological manifestations.

In some embodiments of the first and/or second implantation, thephysiological parameter cannot be computed independently from any other(or subset smaller than the full set) of the respectivesub-physiological parameters. The physiological parameter is computedfrom the combination of the full set of sub-physiological parametersfrom the full set of datasets of sensor. An accuracy of thephysiological parameter computed independently from any one of therespective sub-physiological parameters is lower than a threshold,and/or lower than the physiological parameter computed from thecombination of sub-physiological parameters. The accuracy of thephysiological parameter computed from the combination ofsub-physiological parameters may be above the threshold. The combinationof datasets from the multiple sensors increases the accuracy ofcomputing the physiological parameter, and/or enables computing thephysiological parameter, in comparison to using the datasetsindependently and/or using fewer datasets than used in the combination.Optionally, when the physiological parameter indicates a likelihood of adiagnosis of the physiological pathology, for example, likelihood ofrespiratory pathology such as due to being infected with the viralinfection such as COVID-19, none of the sub-physiological parameterswhen used independently are indicative of likelihood of the diagnosis.The combination of the sub-physiological parameters increases theaccuracy and/or enables the computation of the physiological parameter.

At least some implementations of the systems, methods, apparatus, and/orcode instructions described herein address the technical problem ofrapid and/or accurate screening of people for likelihood ofphysiological pathology. At least some implementations of the systems,methods, apparatus, and/or code instructions described herein improvethe technology of rapid and/or accurate screening of people forlikelihood of physiological pathology.

Likelihood of physiological pathology may be detected, for example,using contact sensors that provide highly accurate measurements. Sincesuch contact sensors are highly accurate, only a single sensor type isused to acquire the data. For example, a thermometer placed under thetongue of the person that measures temperature, a pulse oximeter sensorplaced on a finger of a person that measures heart rate and/or oxygensaturation (SpO2), and/or wearable contact sensors that measurerespiratory rate and/or heart rate. There are several technical problemswith such accurate contact sensors. One example of a problem is thatsuch contact sensors are irrelevant for rapid screening of people forphysiological pathology, due to lack of time to acquire a properreading, infection and safety issues from reuse of contact sensors fordifferent people, and privacy issues in forcing people to undergocontact based measurements. For example, checking people suffering froma viral disease, such as COVID-19, during an outbreak. Non-contacttemperature sensors are routinely used, for example, to screen peopleentering a shopping mall, boarding flights, entering other enclosures,and/or cars entering parking areas and/or other driving zones. Suchnon-contact temperature sensors are inaccurate in measuring temperature.Moreover, COVID-19 presents differently in different people, such that aperson with no apparent fever may be infected with COVID-19, make somemeasurements such as temperature inaccurate in detecting people infectedwith COVID-19.

There are other technical problems with other sensors used for detectinglikelihood of physiological pathology. First, as discussed above, somephysiological pathologies, for example respiratory pathology such asCOVID-19, may present with a different constellation of symptoms indifferent people. So screening for such respiratory pathologies such asCOVID-19 using a single sensor is inherently accurate in that the singlesensor will miss those that are infected but not presenting the symptomsensed by the sensor, such as fever. Second, non-contact sensors tend tobe less accurate than contact sensors, for example, due to errors in theremotely acquired data, interference from other sources (e.g., affectedby strong electromagnetic (EM) signals generated from an interferingsource), and/or situations where such sensors do not work well (e.g.,person wearing thick and/or dark jacket distorts temperature data ofthermal images of the body underneath the jacket). Third, the time toacquire sufficient data by a certain sensor for an accurate measurementmay be long. For example, a video of thermal images may require greaterthan 30 seconds, or more than 1 minute or longer, in order to havesufficient data to estimate a certain physiological parameter. Suchdelay may be unacceptable, for example, for screening passengers loadingonto a plane, or entering a movie theater, in particular when the lineupis long.

At least some implementations of the systems, methods, apparatus, and/orcode instructions described herein provide solutions to one or more ofthe above mentioned technical problems, by using multiple sensors,optionally non-contact, optionally of different modalities, that capturerespective dataset, optionally simultaneously. For example, a thermalsensor (e.g., IR camera) captures thermal images at the same time as aradar sensor captures motion displacement data. The outputs of thesensors are fused, i.e., a combination of the datasets, and/or acombination of sub-physiological parameters extracted from respectivedatasets, is analyze to compute a physiological parameter indicative ofphysiological pathology. The fusion of the outputs of the sensors, whereeach sensor outputs a different dataset based on a different modality,and a different sub-physiological parameter is extracted from eachdataset, enables detection of likelihood of physiological pathology inpatients where the constellation of symptoms differs between patients.For example, respiratory rate is estimated from thermal images,temperature is estimated from the thermal images, heart rate isestimated from the Doppler and/or radar sensor, and oxygen saturation isestimated from a SWIR and/or RGB sensor. The fusion of data from themultiple sensors may improve likelihood and/or accuracy of detectingCOVID-19 and/or other physiological pathologies that present withdifferent combinations of symptoms in different people. Alternatively oradditionally, in another implementation, fusion of the outputs of thesensors, where each sensor outputs a different dataset based on adifferent modality, and the same sub-physiological parameter isextracted from each dataset, enables improved accuracy of measurement ofa single physiological parameter, which may indicate likelihood of thephysiological pathology. For example, respiratory rate is measured byfusion of datasets acquired by a radar sensor, and a thermal sensor. Therespiratory rate measured by the fusion of the datasets is more accuratethan the respiratory rate measured only by the radar sensor, and/or moreaccurate than the respiratory rate measured only by the thermal sensor.Moreover, the time required for the radar sensor and the thermal sensorto obtain enough data for accurate measurement of the respiratory rate,when the data of the two sensors is fused, may be sufficiently shortedthat the amount of time required for only one of the two sensors toobtain enough data for accurate measurement of the respiratory rate.Further, in some cases, data acquired by only one of the sensors may beinsufficient for computing the physiological parameter (e.g.,respiratory rate), at all. In such a case, fusion of data from two ormore sensors is required to compute the physiological parameter. It isnoted that the two implementations may be combined, for example, usingtwo or more different sensors to measure two or more differentsub-physiological parameters, and using at least two of the sensors toobtain a more accurate measure of one of the sub-physiologicalparameters.

Before explaining at least one embodiment of the invention in detail, itis to be understood that the invention is not necessarily limited in itsapplication to the details of construction and the arrangement of thecomponents and/or methods set forth in the following description and/orillustrated in the drawings and/or the Examples. The invention iscapable of other embodiments or of being practiced or carried out invarious ways.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, and any suitable combination of theforegoing. A computer readable storage medium, as used herein, is not tobe construed as being transitory signals per se, such as radio waves orother freely propagating electromagnetic waves, electromagnetic wavespropagating through a waveguide or other transmission media (e.g., lightpulses passing through a fiber-optic cable), or electrical signalstransmitted through a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

Reference is now made to FIG. 1, which is a flowchart of a method formeasuring a physiological parameter of a person indicative ofphysiological pathology, in accordance with some embodiments of thepresent invention. Reference is also made to FIG. 2, which is a blockdiagram of components of a system 200 for measuring a physiologicalparameter of a person indicative of physiological pathology, inaccordance with some embodiments of the present invention. System 200may implement the features of the method described with reference toFIG. 1, by one or more hardware processors 202 of a computing device 204executing code instructions stored in a memory (also referred to as aprogram store) 206.

Computing device 204 may be implemented as, for example, a clientterminal, a server, a virtual machine, a virtual server, a computingcloud, a mobile device, a desktop computer, a thin client, a Smartphone,a Tablet computer, a laptop computer, a wearable computer, glassescomputer, and a watch computer.

Multiple architectures of system 200 based on computing device 204 maybe implemented. In an exemplary implementation, computing device 204storing code 206A may be implemented as one or more servers (e.g.,network server, web server, a computing cloud, a virtual server) thatprovides services (e.g., one or more of the acts described withreference to FIG. 1) to one or more servers 218 and/or client terminals208 over a network 210, for example, providing software as a service(SaaS) to the servers 218 and/or client terminal(s) 208, providingsoftware services accessible using a software interface (e.g.,application programming interface (API), software development kit(SDK)), providing an application for local download to the servers 218and/or client terminal(s) 208, and/or providing functions using a remoteaccess session to the servers 218 and/or client terminal(s) 208, such asthrough a web browser and/or viewing application. For example, users useclient terminals 208 to access computing device 204 to provide thedatasets acquired by the multiple sensors, and/or view and/or receivethe physiological parameter indicative of physiological pathology. Forexample, using an installed application and/or by using a web browser toconnect to computing device 204, and/or communicating data withcomputing device 204 using a software interface (application programminginterface (API) and/or software development kit (SDK). In anotherexample, computing device 204 is a standalone system, for example, alaptop connected to sensors 212 and running locally stored code 206A.

Computing device 204 receives datasets acquired by multiple sensors 212,optionally non-contact remote sensors. Examples of sensors 212 include:a thermal sensor capturing thermal images of radiation that correlateswith temperature (e.g., in the long-infrared (IR) range of theelectromagnetic spectrum (e.g., about 9000-14000 nanometers)), InGaAssensors, FPA sensors, short-wave infrared (SWIR) sensors, near infrared(NIR) sensors, standard visible light sensors (e.g., CCD and/or CMOSsensors, such as red, green, blue (RGB) sensors), radar sensors,ultrasound sensors, laser speckle imaging, Lidar, and biometric sensors.Sensor 212 may include acoustic sensors. Sensor 212 may transmitacquired datasets to computing device 204, for example, via a directconnected (e.g., local bus and/or cable connection and/or short rangewireless connection), and/or via a network 210 and a network interface222 of computing device 204 (e.g., where sensors are connected viainternet of things (IoT) technology and/or are located remotely from thecomputing device).

Network interface 222 may be implemented as, for example, a wireconnection (e.g., physical port), a wireless connection (e.g., antenna),a network interface card, a wireless interface to connect to a wirelessnetwork, a physical interface for connecting to a cable for networkconnectivity, and/or virtual interfaces (e.g., software interface, API,SDK, virtual network connection, a virtual interface implemented insoftware, network communication software providing higher layers ofnetwork connectivity).

Memory 206 stores code instructions executable by hardware processor(s)202. Exemplary memories 206 include a random access memory (RAM),read-only memory (ROM), a storage device, non-volatile memory, magneticmedia, semiconductor memory devices, hard drive, removable storage, andoptical media (e.g., DVD, CD-ROM). For example, memory 206 may code 206Athat execute one or more acts of the method described with reference toFIG. 1.

Computing device 204 may include data storage device 220 for storingdata, for example, datasets(s) 220A acquired by sensors 212, asdescribed herein. Data storage device 220 may be implemented as, forexample, a memory, a local hard-drive, a removable storage unit, anoptical disk, a storage device, a virtual memory and/or as a remoteserver 218 and/or computing cloud (e.g., accessed over network 210). Itis noted that dataset(s) 220A may be stored in data storage device 220,for example, with executing portions loaded into memory 206 forexecution by processor(s) 202.

Computing device 204 and/or client terminal(s) 208 include and/or are incommunication with one or more physical user interfaces 224 that includea mechanism for entering data and/or viewing data, for example, atouchscreen display used to indicate a new person for analysis, and/orfor presenting the computed physiological parameter Exemplary userinterfaces 224 include, for example, one or more of, a touchscreen, adisplay, a keyboard, a mouse, and voice activated software usingspeakers and microphone.

Server(s) 218 may receive instructions generated by computing device204. For example, server 218 may be implemented as a controller of anautomated door that is automatically opened based on the instructions inresponse to the physiological parameter indicative of non-likelihood ofphysiological pathology, and is automatically closed and/or locked basedon the instructions in response to the physiological parameterindicative of likelihood of physiological pathology.

Referring now back to FIG. 1, at 102, respective datasets are receivedfrom each one of multiple remote, non-contact sensors monitoring aperson. Each respective dataset may be the output of the respectivesensor, for example, a video including multiple frames, sequentiallyacquired individual still frames, and respective values indicative ofother measurements made by the respective sensor. The respective datasetmay be the raw output of the respective sensor, and/or output that hasbeen processed to obtain a value. For example, raw signals thatcorrespond to displacement of the chest of the person, or a valueindicating the amount of displacement of the chest.

The datasets may include an acoustic dataset obtained from an acousticrecording.

The respective datasets are simultaneously received from the multiplesensors that simultaneously acquire and/or output the respectivedataset.

The respective datasets are simultaneously received over a timeinterval. In the first implementation where the respective datasetacquired by each respective sensor represents a different physiologicalmanifestation of the person, the time interval may be long enough toenable sufficient amount of data in each respective dataset to computethe respective physiological manifestation, for example, at an accuracyabove a threshold. In the second implementation where the respectivedataset acquired by each respective sensor represents a similar (e.g.,same) physiological manifestation of the person, the time interval maybe short enough so that the data in each respective dataset isinsufficient to compute the physiological manifestation at an accuracyabove a threshold. The time interval may be shorter than a time intervalof each respective dataset required to compute each respectivesub-parameter with an accuracy similar to an accuracy of thephysiological parameter.

The time interval may be too short for using only one sensor, or fewerthan the total number of sensors, to accurate compute the physiologicalmanifestation at an accuracy above the threshold. As described herein,in at least some implementations, using the combination of datasets fromthe multiple sensors, even when obtained over a time interval that istoo short for using each dataset independently, provides for computingthe physiological manifestation at the accuracy above the threshold.

At 103, the respective datasets may be analyzed for errors. Optionally,the combination of time synchronized dataset are analyzed for errors.The detected errors may be corrected and/or the errors may be eliminatedin the respective datasets. Alternatively or additionally, the datasetwith identified errors is excluded from extraction of certainsub-physiological parameters. The dataset with identified errors may beused to extract other sub-physiological parameters which may beunaffected by the error. The correction and/or removal of the errorsand/or exclusion of the dataset may improve reliability and/or accuracyof the subsequently extracted sub-physiological parameter(s) and/orcomputation of the physiological parameter.

The errors may be detected, for example, by correlating synchronizedportions of the datasets to each other, and identifying non-correlatedportion of the datasets, for example, when a correlation value falls tobelow a threshold, and/or when a correlation value for the correlateddatasets falls more than a predefined amount of a certain portion of thedatasets. In another example the errors may be detected by computingfeatures of the datasets and comparing the features to predefinednon-error and/or error values (e.g., using a set of rules), for example,maximum value, standard deviation, expected patterns, and the like.Features may be extracted for each dataset alone, and/or for acombination of the datasets (e.g., sum of two datasets, differentbetween two datasets, peaks/troughs in both dataset) In another example,the errors may be detected by feeding the datasets (each alone, and/orin a combination) into an error classifier trained on datasets obtainedfrom other people using similar sensors, and labelled with an indicationof error or non-error.

Some examples of removing and/or correcting errors are provided.

In one example, selective elimination/removal of some of the sensordatasets is performed when such datasets are deemed irrelevant. Forexample, when the person wears a thick sweater/coat, a first sensor(e.g., used to provide the dataset from which heart rate and/orrespiratory rate sub-physiological parameters are extracted) aimed atthe chest may generate irrelevant dataset from which the heart rateand/or respiratory rate cannot be extracted (e.g., especially for heartrate) while another sensor aimed at the carotid artery at the throat orat the face may still output a dataset from which heart rate may beextracted. In such a case, the first dataset of the first sensor may beremoved from further processing. The removal may eliminate errors due tothe processing of the irrelevant first dataset. In another example theperson may have a thick beard, a scarf, and the chest area. In such acase, the second dataset is removed, and the first dataset is used.

In another example, human motion (intentional and/or unintentional)during the measurement may affect some sensors datasets more thanothers. For example, motion may affect the reliability of computing therespiratory rate sub-physiological parameter more than that of computingthe heartbeat/temperature/oximetry sub-physiological parameter. Thedataset may be analyzed to detect motion (e.g., analyzes imagesoutputted by RGB/IR image sensors or other sensors to detect excessivemotion of the person). The dataset with motion may be excluded fromcomputation of the respiration parameter sub-physiological parameter.

In yet another example, electronic instrumentation may affect radarsensors. The dataset outputted by the radar sensor may be analyzed todetermine whether the dataset is affected by electronic instrumentation.The errors may be corrected and/or removed.

In yet another example, the time synchronized datasets (e.g., timeseries) are analyzed as a combination. For example, the air outflow fromthe nostrils/mouth, which may be computed from a first dataset of afirst sensor, is expected to correlate with chest contraction/distanceincrease from a second dataset of a second sensor that is timesynchronized with the first sensor. Errors may be detected whencorrelation is poor, for example, below a threshold. Such errors may beremoved from further processing. When correlation is good, for example,above a threshold, the time synchronized datasets represent high qualityand/or accurate measurements. The validity and/or quality of the datamay be enhanced significantly by intelligently combining the datasets inthe time-series form. It is noted that the second dataset of the sensormay not necessarily correlate with the time synchronized first datasetof the first sensor.

In yet another example, a combination of the time synchronized datasetsmay help detect errors, and/or improve extraction of somesub-physiological parameters. For some sensors, there may significant‘crosstalk’ between the various datasets outputted by the sensors. Forexample, breathing and heartbeat extracted sub-physiological parametersof the person may correspond in some areas and in some sensors to thesame physical measurement (e.g., distance of chest/throat) obtained fromthe same dataset. When the Breathing frequency sub-physiologicalparameter may be considered omega_1 (e.g., 1/15 of hertz (Hz)) and theheartrate sub-physiological parameter may be considered omega_2 (e.g., 1Hz), the following frequencies with similar amplitude may be detected:omega 2+omega_1, omega_2-omega_1, omega_2. Having a separate estimatefrom another sensor of either omega_1 or omega_2 may help isolate thevalues of omega_1 and omega_2.

In yet another example, at least one of the respective datasets obtainedfrom at least one of the remote non-contact sensors is analyzed forvalidating that a set of rules is met. In such implementation, theanalysis for errors may be, for example, to determine when the set ofrules is not met (i.e., error) or when the set of rules is met (i.e., noerror). Alternatively or additionally, other data obtained from othersensors is analyzed to determine whether the set of rules is met. Whenthe set of rules is met, the remaining features of the method describedwith reference to FIG. 1 may be implemented, for example, one orfeatures described with reference to 104-124 of FIG. 1. When the set ofrules is not met, an indication may be generated, for example, a message(e.g., text message presented on a display, an audio recording playedover speakers, a video played on the display) indicating what theproblem is and/or how to fix the problem in order to meet the set ofrules. The set of rules may be, for example, to evaluate whether thedata captured of a subject in a vehicle is reliable for processing.Exemplary set of rules include one or more of: the subject is in thevehicle, the window of the vehicle is open, a location of the subjectand/or vehicle is according to a target location (e.g., subject is inthe front seat, vehicle positioned in front of sensor to enable sensorto capture data), an engine of the vehicle is turned off, and vibrationsof the vehicle are below a threshold. Examples of instructions that aregenerated when the set of rules is not met may include one or more of:subject should sit still in vehicle and look towards the sensor(s), openwindow, move car forward/reverse relative to sensor(s), turn engine off,vehicle vibrating too much—turn off engine and/or subject to exitvehicle.

At 104, a respective sub-physiological parameter is extracted from eachrespective dataset of each respective sensor.

According to the first implementation, different respectivesub-physiological parameters are extracted from respective datasets.Examples of sub-physiological parameters and corresponding remotenon-contract sensors from which respective datasets used to compute thecorresponding respective sub-physiological parameter include:

-   -   Temperature computed from an analysis of thermal images captured        by a thermal sensor capturing thermal images.    -   Respiratory rate and/or breathing pattern computed from an        analysis of data outputted by a sensor, for example, a thermal        sensor, a radar sensor, a Doppler sensor, and a short wave        infrared sensor (SWIR) sensor.    -   Heart rate and/or heartbeat pattern computed from an analysis of        data outputted by a sensor, for example, a radar sensor, a        Doppler sensor, a thermal sensor, and a visual light sensor.    -   Oxygen saturation (SpO2) computed from an analysis of data        outputted by a sensor, for example, a SWIR sensor, and a visual        light sensor.    -   Nasal congestion, sore throat, hoarse voice, cough, and the        like, obtained from an analysis of an acoustic dataset outputted        by an acoustic sensor.

According to the second implementation, the same and/or similarsub-physiological parameter is extracted from each respective dataset.Examples of sub-physiological parameters representing the same and/orsimilar single physiological manifestation indicative of physiologicalpathology include: respiratory rate and/or breathing pattern, heart rateand/or heartbeat pattern, and blood oxygen saturation (SpO2). Examplesof different datasets and corresponding remote non-contract sensors fromwhich the same and/or similar sub-physiological parameter is computedinclude thermal images acquired by a thermal sensor, near infrared (NIR)images acquired by a NIR sensor, visual light images acquired by avisual light sensor, and a dataset indicative of chest motion capturedby a Doppler sensor and/or radar sensor.

Optionally, one of the sub-physiological parameters is a breathingpattern indicative of breathing of the person. In some embodiments, oneof the remote non-contact sensors may be a thermal sensor capturing asequence of thermal images depicting a face of the person. The breathingpattern may be computed by analyzing changes in pixel intensity valuesof pixels, which are indicative of heating and cooling, corresponding tonostrils and/or mouth cavity (i.e., of an open mouth) and/or face maskof the person in the sequence of thermal images. The change in pixelintensity values corresponding to the nostril and/or mouth cavity and/orface mask may be performed by analyzing the average of pixel intensityvalues of pixels depicting the face mask and/or nostrils and/or mouthcavity, for example, in segmented regions of the mask and/or face. Thechange in pixel intensity values of pixels corresponding to the facemask may exclude pixels of the face mask corresponding to the nose ofthe person. The heating and/or cooling pattern of the nose may bedifferent than the rest of the mask, and not necessarily correlated withbreathing pattern, since the nose may heat up and remain at asubstantially constant temperature thereafter. The heating and/orcooling of the face mask that excludes the nose may correlate withbreathing patterns, indicative of cooling of the mask during inhalationand heating of the mask during exhalation. In some embodiments, thenostrils and/or mouth cavity and/or mask of the person in the sequenceof thermal images may be segmented, for example, by identifying regionsof changes in pixel intensity values. The nostrils and/or mouth cavityaren't visible in the image and/or the nostrils and/or mouth cavitycannot be accurately detected (e.g., above a threshold) in the image,for example, when the person is properly wearing a mask covering thenose and mouth, and/or when the person is looking downwards. In such acase, the breathing pattern may be computed by analyzing changes inpixel intensity values of pixels corresponding to a region on the faceof the person under the nose of the person, for example, the lips, thephiltrum, and nasolabial sulcus.

Optionally, one or more of the sub-physiological parameters may bedemographic parameters of the person, for example, age, gender, weight,and/or height. The demographic parameters may be obtained by an analysisof a single images or one or more images captured by sensors, forexample, infrared and/or RBG image sensors. The demographic parametersmay be obtained, for example, by feeding the respective image into ademographic classifier trained on a training dataset of images ofdifferent people captured by the sensor, labelled with an indication ofthe demographic parameter of the person depicted in the image. Otherapproaches include, for example, performing physical measurements of theperson from the images, such as counting the number of pixels the personspans in the image and based on the calibration of distance per pixel,the height and/or weight may be estimated (e.g., using a weight estimateformula).

Reference is now made to FIG. 3, which is a graph 302 for computation ofsub-physiological parameter(s) generated from an analysis of multiplethermal images of a person, in accordance with some embodiments of thepresent invention. Y-axis 304 is a scale (e.g., normalized pixelintensity values) depicting a thermal reading indicative of temperatureat an area below a nose of the person. It is noted that the temperaturemay be obtained at other locations described herein, for example,nostrils, inside a mouth cavity when the mouth is open, and regions of amask being worn over the mouth and nose. X-axis 306 is an indication oftime and/or sequence number of the images. Graph 302 is generated byplotting the respective temperature for the respective thermal imageframe at the sequence number of the respective frame. Thesub-physiological parameter, for example, respiratory rate may becomputed, for example, based on local maximal temperature values, forexample, a time between local maximal peak 308A and local maximal peak308B indicates the time for one inspiratory/respiratory cycle. Therespiratory rate may be obtained as the number of suchinspiratory/respiratory cycles over one minute, or other measures.

Reference is now made to FIG. 4, which includes an example of a thermalimage 402 of a chest used to compute sub-physiological parameter(s), inaccordance with some embodiments of the present invention. The thermalimage may be obtained while the person is wearing the shirt (or othergarment, or a blanket) or without the shirt. The thermal images of thechest may be analyzed to identify heating/cooling patterns. Thesub-physiological parameter(s), for example, respiratory rate, may becomputed based on the heating/cooling pattern, for example, bygenerating a graph of an indication of temperature as a function offrame sequence number (and/or time) as described with reference to FIG.3, and/or other approaches described herein.

Reference is now made to FIG. 5, which is an example of a graph 502created based on an output of a non-contact radar sensor sensing a chestof a person correlated with another graph 504 created based on output ofa contact belt sensor measuring the chest of the person, in accordancewith some embodiments of the present invention. Graph 502 indicates thatthe non-contact sensor accurate generates an indication of movement of achest (e.g., chest displacement) which may be used for computingsub-physiological parameter(s) such as respiratory rate and/or heartrate.

Reference is now made to FIG. 6, which includes thermal images 602 and604 that are analyzed to measure a temperature of an identified mask 606(measured at 35.4 degrees Celsius) and/or measure a temperature of anidentified forehead 608 (measured at 36.1 degrees Celsius) of therespective depicted person, in accordance with some embodiments of thepresent invention. The mask and/or forehead may be segmented. Changes inthe temperature of the mask may be analyzed to identify breathingpatterns, as described herein.

Referring now back to 104 of FIG. 1, the thermal images may be analyzedto identify one or more facial features and/or mask features indicativeof regions depicting parts of a face of the person, for example, whenthe person is depicted as wearing a mask, bulges of the mask created bythe nose underneath may be detected.

One or more sub-physiological parameters (e.g., breathing pattern) maybe computed by analyzing changes in pixel intensity values of pixelscorresponding to nostrils and/or face mask and/or open mouth of theperson according to the identified facial features and/or mask features.For example, when the person is not wearing a mask, the breathingpattern is computed based on changes in pixel intensity values of thepixels corresponding to the nostrils and/or open mouth. In anotherexample, when the person is wearing a mask, the breathing pattern iscomputed based on changes in pixel intensity values of regions of theface mask.

The change in pixel intensity values varies by an amount correspondingto a temperature change range. A rate of change of the pixel intensityvalues corresponds to a candidate breathing rate range. The breathingpattern may be computed as a breathing rate based on a time intervalfrom maximal to minimal (or maximal to maximal, or minimal to minimal,or minimal to maximal) pixel intensity values of pixels corresponding tothe nostril and/or face mask of the sequence of thermal images. Forexample, certain regions of pixels of the mask of the person cyclebetween high and low pixel intensity values, indicating the breathingpattern of the person, breathing in cool air and exhaling warm air. Thepeak to peak time interval (e.g., peak pixel intensity time to anotherpeak pixel intensity time) may represent the time between eachexhalation. The time for each breath may be estimated as the minimalpeak time indicating breathing in to maximal peak time indicatingbreathing out.

Reference is now made to FIG. 7, which includes thermal images 702depicting a person walking, for which facial features are identified,represented by stars 704, in accordance with some embodiments of thepresent invention. Sub-physiological parameters may be computed based onthe detected facial features as described herein. Image 706 depicts onesample image, and a corresponding annotated image 708 with identifiedfacial features. 710.

At 106, a combination of the sub-physiological parameters is analyzed.Alternatively or additionally, a combination of the datasets isanalyzed.

Optionally, the combination of the sub-physiological parameters isanalyzed according to the demographic parameter(s) of the person. Thedemographic parameter(s) may define a baseline from which othersub-physiological parameters may be determined to be normal or abnormal.For example, the breathing rate, heart rate, and/or temperature rangesconsidered as non-pathological (or pathological) for an obese 65 yearold tall person are different than the breathing rate, heart rate,and/or temperature ranges for a short 15 year old, thin, short, athleticchild. According to the first implementation, each sub-physiologicalparameter is different. Alternatively, according to the secondimplementation, each sub-physiological parameter is similar and/or thesame.

The analysis of the combination of sub-physiological parameters and/orcombination of datasets may be performed, for example, by applying a setof rules, computing a correlation between the datasets and/or thesub-physiological parameters, and/or inputting the combination of thesub-physiological parameters into a classifier trained on a dataset ofcombinations of sub-physiological parameters and a label of thephysiological parameter indicative of physiological pathology.

As used herein, the term classifier may refer, for example, to astatistical classifier and/or machine learning model that maps inputs toan outcome. The classifier(s) described herein may be implemented, forexample, to one or more neural networks of various architectures (e.g.,artificial, deep, convolutional, fully connected), support vectormachine (SVM), logistic regression, k-nearest neighbor, decision trees,and combinations of the aforementioned

For example, according to the first implementation, likelihood of beinginfected with COVID-19 (or another viral illness) is determined based ona set of rules that evaluate multiple sub-physiological parameterscomputed from different datasets. For example, when temperature of aperson's forehead obtained from a thermal image is above a firstthreshold, and a respiratory rate computed from a radar sensor measurechest motion of a chest of the person is above a second threshold, andan oxygen saturation level measured by an RGB and/or SWIR sensor isbelow a third threshold, likelihood of viral infection is detected.

For example, according to the second implementation, a value indicativeof an amount of correlation between the datasets may be computed. Forexample, to compute respiratory rate, a graph of pixel intensity valuesof nostrils obtained from thermals images may be correlated with a graphof respiratory motion obtained from output of a radar sensor. At timeswhen the two graphs have a correlation above a threshold, therespiratory rate may be computed from the correlated graphs (or from oneof the two graphs during the time when the graphs are correlated).

At 108, the physiological parameter indicative of physiologicalpathology is computed according to the analysis. The physiologicalparameter may be, for example, likelihood of the subject being infectedwith the viral disease and/or likelihood of the subject suffering fromphysiological pathology. The physiological parameter may be represented,for example, as a value between 0-1 (or 0-100), a binary indication(e.g., yes/no), or one of multiple classification categories (e.g.,none, mild, severe).

The physiological parameter may be computed, for example, by a mappingdataset that maps values of the sub-physiological parameters into thephysiological parameter (e.g., in a multi-dimensional space), afunction, and/or a classifier trained on a training dataset of sampledatasets and/or combinations of sub-physiological parameters labelledwith the physiological parameter. For example, the value of thephysiological parameter is computed based on the values of thesub-physiological parameters, and/or number of rules met. For example,the higher the temperature, and the higher the respiratory rate, and thelower the oxygen saturation, the more likely that the person is infectedwith the viral illness.

In some embodiments, the extracting of the extracting of thesub-physiological parameters, and/or the analysis of the combination ofthe sub-physiological parameters, and/or the computing of thephysiological parameter is performed by inputting the datasets based onthe outputs of the different sensors into a classifier. Thephysiological parameter is obtained as an outcome of the classifier. Theclassifier is trained on a training dataset including, for each ofmultiple subjects, a respective dataset acquired by each of the remotenon-contact sensors, and an associated label of the physiologicalparameter indicative of physiological pathology. In another example, thetraining dataset includes, for each of multiple subjects, thecombination of sub-physiological parameters and the associated label ofthe physiological parameter.

At 110, instructions may be automatically generated according to thephysiological parameter. The instructions may be, for example, codeand/or other signals for execution by a controller. The controller maybe, for example, for automatic opening and/or closing of a door forentry to an enclosure such as an office building, subway, airport, mall,and a movie theater. The door may be instructed to open for admission tothe person when the physiological parameter indicates unlikelihood ofphysiological pathology, for example, unlikely to be infected with theviral disease. The door may be instructed to close and/or to maintainthe door in the closed state to prevent admission of the person when thephysiological parameter indicates likelihood of physiological pathology,for example, when the person is likely infected with the viral disease.

In another example, the subject may be sitting in a vehicle (e.g., car).The datasets (e.g., as in 102) depict the subject in the vehiclecaptured with a window of the vehicle open. In another example, wherethe car window is closed, the analysis may be performed for closedwindows, for example, training classifiers on datasets of subjects incars with closed windows. Instructions are generated for admitting thevehicle to a parking area and/or to let the vehicle keep on driving whenthe physiological parameter is below a threshold, optionallyautomatically activating a mechanism to open a gate and/or generating anindication for a user to manually open the gate. When the physiologicalparameter is above the threshold, the gate may remain closed, denyingthe vehicle admission.

In other examples, the instructions may be, for example, to trigger analert to a user. For example, when a person with the viral disease is inhome isolation and being remotely monitored for respiratory difficultiesdue to the viral disease. The alert may be generated, for example, as atext message, a phone call, an email, and/or a pop-up message on amobile device and/or administrative server of the user, warning that thecondition of the person being monitored is becoming more severe.

At 112, an effective treatment may be administered for treatment of thephysiological pathology and/or the person may be sent for additionaltesting. For example, when the physiological pathology is COVID-19and/or shortness of breath, a treatment effective for COVID-19 and/orshortness of breath is administered, for example, supplemental oxygen,antibiotics, anti-viral, mechanical ventilation, bronchodilators, andcorticosteroids. In another example, the person may be sent foradditional testing, for example, x-ray, ECG, CT scan, pulmonaryexamination, and the like.

Optionally, at 114, prior to extraction of the sub-physiologicalparameter as in 104, a first dataset captured by a first sensor isanalyzed for calibrating and/or directing the second sensor for thesimultaneous capture of the second dataset, as described with referenceto 102.

In an example, the first remote non-contact sensor is a thermal and/orvisual sensor that captures a sequence of thermal and/or visual imagesdepicting a chest and/or head of the person. The second remotenon-contact sensors is a Doppler and/or radar sensor. One or morethermal and/or visual images are analyzed to identify a target locationof the chest and/or head of the person depicted therein.

At 116, based on the analysis of the first dataset, the second sensor iscalibrated. The values of the first dataset are used to adjust thesecond sensor to obtain an improved second dataset. The combination ofthe first and second datasets may improve the accuracy of thephysiological parameter.

Optionally, instructions are generated for automatic adjustment of asteering mechanism for adjusting an orientation of the second sensor(e.g., Doppler and/or radar sensor) for capturing the second datasetaccording to the target location identified in the first dataset (i.e.,the thermal images).

At 118, one or more of 102-112 are implemented. At 102, the first andsecond datasets are then simultaneously received from the first sensorand the calibrated and/or directed second sensor. At 104, a firstsub-physiological parameter is extracted from the first dataset acquiredby the first sensor (e.g., thermal and/or visual sensor), and a secondsub-physiological parameter is extracted from the second datasetacquired by the second sensor (e.g., Doppler and/or radar sensor).

At 120, during (e.g., before, after, during, simultaneously with) 102, acertain dataset acquired by a certain remote non-contact sensor isanalyzed to obtain tracked locations on multiple fixed points on thehead of the person. The certain sensor may be a camera (e.g., thermal,visual) that generates thermal and/or visual images. The images areanalyzed to obtain the tracked locations.

At 122, an additional dataset depicting the head of the person acquiredby another remote non-contact sensor, which is different than thecertain sensor, is received. The additional dataset is corrected usingthe fixed points of the certain dataset, for tracking the fixed pointson the head of the person depicted in the additional dataset.

At 124, one or more of 104-112 are implemented using the correctedadditional dataset.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

It is expected that during the life of a patent maturing from thisapplication many relevant sensors will be developed and the scope of theterm sensor is intended to include all such new technologies a priori.

As used herein the term “about” refers to ±10%.

The terms “comprises”, “comprising”, “includes”, “including”, “having”and their conjugates mean “including but not limited to”. This termencompasses the terms “consisting of” and “consisting essentially of”.

The phrase “consisting essentially of” means that the composition ormethod may include additional ingredients and/or steps, but only if theadditional ingredients and/or steps do not materially alter the basicand novel characteristics of the claimed composition or method.

As used herein, the singular form “a”, “an” and “the” include pluralreferences unless the context clearly dictates otherwise. For example,the term “a compound” or “at least one compound” may include a pluralityof compounds, including mixtures thereof.

The word “exemplary” is used herein to mean “serving as an example,instance or illustration”. Any embodiment described as “exemplary” isnot necessarily to be construed as preferred or advantageous over otherembodiments and/or to exclude the incorporation of features from otherembodiments.

The word “optionally” is used herein to mean “is provided in someembodiments and not provided in other embodiments”. Any particularembodiment of the invention may include a plurality of “optional”features unless such features conflict.

Throughout this application, various embodiments of this invention maybe presented in a range format. It should be understood that thedescription in range format is merely for convenience and brevity andshould not be construed as an inflexible limitation on the scope of theinvention. Accordingly, the description of a range should be consideredto have specifically disclosed all the possible subranges as well asindividual numerical values within that range. For example, descriptionof a range such as from 1 to 6 should be considered to have specificallydisclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from2 to 4, from 2 to 6, from 3 to 6 etc., as well as individual numberswithin that range, for example, 1, 2, 3, 4, 5, and 6. This appliesregardless of the breadth of the range.

Whenever a numerical range is indicated herein, it is meant to includeany cited numeral (fractional or integral) within the indicated range.The phrases “ranging/ranges between” a first indicate number and asecond indicate number and “ranging/ranges from” a first indicate number“to” a second indicate number are used herein interchangeably and aremeant to include the first and second indicated numbers and all thefractional and integral numerals therebetween.

It is appreciated that certain features of the invention, which are, forclarity, described in the context of separate embodiments, may also beprovided in combination in a single embodiment. Conversely, variousfeatures of the invention, which are, for brevity, described in thecontext of a single embodiment, may also be provided separately or inany suitable subcombination or as suitable in any other describedembodiment of the invention. Certain features described in the contextof various embodiments are not to be considered essential features ofthose embodiments, unless the embodiment is inoperative without thoseelements.

Although the invention has been described in conjunction with specificembodiments thereof, it is evident that many alternatives, modificationsand variations will be apparent to those skilled in the art.Accordingly, it is intended to embrace all such alternatives,modifications and variations that fall within the spirit and broad scopeof the appended claims.

All publications, patents and patent applications mentioned in thisspecification are herein incorporated in their entirety by referenceinto the specification, to the same extent as if each individualpublication, patent or patent application was specifically andindividually indicated to be incorporated herein by reference. Inaddition, citation or identification of any reference in thisapplication shall not be construed as an admission that such referenceis available as prior art to the present invention. To the extent thatsection headings are used, they should not be construed as necessarilylimiting. In addition, any priority document(s) of this applicationis/are hereby incorporated herein by reference in its/their entirety.

What is claimed is:
 1. A system for measuring a physiological parameterof a person indicative of physiological pathology, comprising: aplurality of remote non-contact sensors, each of a different type ofsensing modality; at least one hardware processor executing a code for:simultaneously receiving over a time interval, from each of theplurality of remote non-contact sensors monitoring a person, arespective dataset; extracting, from each respective dataset, arespective sub-physiological parameter of a plurality ofsub-physiological parameters; analyzing a combination of the pluralityof sub-physiological parameters; and computing a physiological parameterindicative of physiological pathology according to the analysis, whereinan accuracy of the physiological parameter computed from the combinationis higher than an accuracy of the physiological parameter independentlycomputed using any one of the plurality of sub-physiological parameters.2. The system of claim 1, further comprising code for: prior to theextracting: detecting errors according to an analysis of a combinationof the datasets when time synchronized; correcting and/or removing thedetected errors from the datasets, wherein the extracting is performedfor each respective dataset for which the detected errors are correctedand/or removed.
 3. The system of claim 1, further comprising code for:prior to the extracting: detecting errors according to an analysis ofeach respective dataset; excluding a certain dataset with detectederrors from extraction of at least one certain sub-physiologicalparameter, wherein the extracting excludes extracting the at least onecertain sub-physiological parameter from the excluded certain dataset.4. The system of claim 1, wherein each sub-physiological parameterdenotes a measurement of a different physiological manifestation of theperson, and the physiological parameter indicates a diagnosis oflikelihood of physiological pathology, wherein none of thesub-physiological parameters when used independently are indicative ofthe diagnosis of likelihood of physiological pathology.
 5. The system ofclaim 4, wherein the plurality of sub-physiological parameters andcorresponding remote non-contract sensors are selected from the groupconsisting of: (i) temperature computed from an analysis of a thermalimages captured by a thermal sensor; (ii) respiratory rate and/orbreathing pattern computed from an analysis of data outputted by asensor selected from the group consisting of: a thermal sensor, a radarsensor, a Doppler sensor, and a short wave infrared sensor (SWIR)sensor; (iii) heart rate and/or heartbeat pattern computed from ananalysis of data outputted by a sensor selected from the groupconsisting of: a radar sensor, a Doppler sensor, a thermal sensor, and avisual light sensor; (iv) oxygen saturation (SpO2) computed from ananalysis of data outputted by a sensor selected from the groupconsisting of: a SWIR sensor, and a visual light sensor; (v) one or moreof: nasal congestion, sore throat, hoarse voice, and cough, obtainedfrom an analysis of an acoustic dataset outputted by an acoustic sensor.6. The system of claim 4, further comprising at least one of: (i)automatically generating instructions to a controller that controls anautomatic door to open the door for admission of the person when thediagnosis indicates unlikelihood of physiological pathology, and toclose the door and/or maintain the door in the closed state to preventadmission of the person when the diagnosis indicates likelihood ofphysiological pathology, and (ii) administering an effective treatmentfor treatment of the physiological pathology, the treatment selectedfrom the group consisting of: supplemental oxygen, antibiotics,anti-viral, mechanical ventilation, bronchodilators, andcorticosteroids.
 7. The system of claim 1, wherein the subject is in avehicle, and the respective datasets depict the subject in the vehiclecaptured with a window of the vehicle open.
 8. The system of claim 7,further comprising code for generating instructions for admitting thevehicle to a parking area when a value of the physiological pathology isbelow a threshold.
 9. The system of claim 1, further comprising code foranalyzing at least one of the respective datasets obtained from at leastone of the plurality of remote non-contact sensors for validating that aset of rules is met, and in response to the set of rules being met,performing the extracting, the analyzing the combination, and thecomputing the physiological parameter, wherein the set of rules includesat least one rule selected from a group consisting of: a subject is in avehicle, a window of the vehicle is open, a location of the subjectand/or vehicle is according to a target location, an engine of thevehicle is turned off, and vibrations of the vehicle are below athreshold.
 10. The system of claim 1, wherein one of the plurality ofsub-physiological parameters comprise a breathing pattern, and one ofthe remote non-contact sensors comprise a thermal sensor capturing asequence of thermal images depicting an open mouth of the person, andfurther comprising code for: computing the breathing pattern byanalyzing changes in pixel intensity values of pixels corresponding to amouth cavity of the person in the sequence of thermal images.
 11. Thesystem of claim 1, wherein each sub-physiological parameter denotes adifferent respective measurement originating from a same singlephysiological manifestation of the person, and the physiologicalparameter indicative of physiological pathology is a single measurementof the same single physiological manifestation.
 12. The system of claim11, wherein a time interval of the datasets used for computing thephysiological parameter is shorter than a time interval of eachrespective dataset required to compute each respective sub-parameterwith an accuracy similar to an accuracy of the physiological parameter.13. The system of claim 11, wherein the same single physiologicalmanifestation indicative of physiological pathology is selected from thegroup consisting of: (i) respiratory rate and/or breathing pattern, (ii)heart rate and/or heartbeat pattern, and (iii) blood oxygen saturation(SpO2), and wherein the datasets and corresponding remote non-contractsensors include two or more sensors selected from the group consistingof: (i) thermal images acquired by a thermal sensor, (ii) near infrared(NIR) images acquired by a NIR sensor, (iii) visual light imagesacquired by a visual light sensor, and a (iv) dataset indicative ofchest motion captured by a Doppler sensor and/or radar sensor.
 14. Thesystem of claim 1, wherein one of the plurality of sub-physiologicalparameters comprise a breathing pattern, and one of the remotenon-contact sensors comprise a thermal sensor capturing a sequence ofthermal images depicting a face of the person, and further comprisingcode for: computing the breathing pattern by analyzing changes in pixelintensity values of pixels corresponding to nostrils and/or face mask ofthe person in the sequence of thermal images.
 15. The system of claim14, further comprising code for segmenting the nostrils and/or mask ofthe person in the sequence of thermal images by identifying regions ofchanges in pixel intensity values, the change in pixel intensity valuesvaries by an amount corresponding to a temperature change range, and arate of change of the pixel intensity values corresponding to acandidate breathing rate range.
 16. The system of claim 14, wherein thebreathing pattern is computed as a breathing rate based on a timeinterval from maximal to maximal pixel intensity values of pixelscorresponding to the nostril and/or face mask of the sequence of thermalimages.
 17. The system of claim 14, wherein analyzing changes in pixelintensity values comprises analyzing an average intensity value ofintensity values of pixels depicting the face mask and/or nostrils. 18.The system of claim 14, wherein the change in pixel intensity values ofpixels corresponding to the face mask excludes pixels of the face maskcorresponding to a nose of the person.
 19. The system of claim 14,wherein the breathing pattern is computed by analyzing changes in pixelintensity values of pixels corresponding to a region on a face of theperson under a nose of the person in the sequence of thermal images. 20.The system of claim 14, further comprising code for analyzing thesequence of thermal images to identify a plurality of facial featuresand/or mask features indicative of regions of each thermal imagedepicting parts of a face of the person, wherein the breathing patternis computed by analyzing changes in pixel intensity values of pixelscorresponding to nostrils and/or face mask of the person according tothe identified plurality of facial features and/or mask features. 21.The system of claim 1, wherein a first of the plurality of remotenon-contact sensors comprises a thermal and/or visual sensor capturing asequence of thermal and/or visual images depicting a chest and/or headof the person, a second of the plurality of remote non-contact sensorscomprises a Doppler and/or radar sensor, and further comprising codefor: analyzing at least one thermal and/or visual image to identify atarget location of the chest and/or head of the person; and generatinginstructions for adjustment of a steering mechanism for adjusting anorientation of the Doppler and/or radar sensor for capturing the datasetfrom the identified target location, wherein a first sub-physiologicalparameter is extracted from a first dataset acquired by the thermaland/or visual sensor, and a second sub-physiological parameter isextracted from a second dataset acquired by the Doppler and/or radarsensor.
 22. The system of claim 1, further comprising code for:analyzing a first dataset acquired by a first remote non-contact sensorto obtain tracked locations on a plurality of fixed points on a head ofthe person; receiving a second dataset depicting the head of the personacquired by a second remote non-contact sensor; and correcting thesecond dataset using the fixed points of the first dataset, for trackingthe plurality of fixed points on the head of the person depicted in thesecond dataset.
 23. The system of claim 1, wherein the extracting andthe analyzing and the computing the physiological parameter comprisesobtaining an outcome of a classifier trained on a training datasetincluding, for each of a plurality of subjects, a respective datasetacquired by each of the plurality of remote non-contact sensors, and anassociated label of the physiological parameter indicative ofphysiological pathology.
 24. The system according to claim 1, wherein atleast one of the plurality of sub-physiological parameters comprises ademographic parameter, and wherein the analyzing the combination of theplurality of sub-physiological is according to a baseline definingphysiological pathology according to the demographic parameter.
 25. Amethod of measuring a physiological parameter of a person indicative ofphysiological pathology, comprising: simultaneously receiving over atime interval, a respective dataset from each of a plurality of remotenon-contact sensors each of a different type of sensing modality thatare monitoring a person; extracting, from each respective dataset, arespective sub-physiological parameter of a plurality ofsub-physiological parameters; analyzing a combination of the pluralityof sub-physiological parameters; and computing a physiological parameterindicative of physiological pathology according to the analysis, whereinan accuracy of the physiological parameter computed from the combinationis higher than an accuracy of the physiological parameter independentlycomputed using any one of the plurality of sub-physiological parameters.26. A computer program product for measuring a physiological parameterof a person indicative of physiological pathology, comprising: anon-transitory memory having stored thereon a code for executing by atleast one hardware processor, comprising instructions for:simultaneously receiving over a time interval, a respective dataset fromeach of a plurality of remote non-contact sensors each of a differenttype of sensing modality that are monitoring a person; extracting, fromeach respective dataset, a respective sub-physiological parameter of aplurality of sub-physiological parameters; analyzing a combination ofthe plurality of sub-physiological parameters; and computing aphysiological parameter indicative of physiological pathology accordingto the analysis, wherein an accuracy of the physiological parametercomputed from the combination is higher than an accuracy of thephysiological parameter independently computed using any one of theplurality of sub-physiological parameters.